Nets with Unreliable Hidden Nodes Learn Error-Correcting Codes
نویسندگان
چکیده
In a multi-layered neural network, anyone of the hidden layers can be viewed as computing a distributed representation of the input. Several "encoder" experiments have shown that when the representation space is small it can be fully used. But computing with such a representation requires completely dependable nodes. In the case where the hidden nodes are noisy and unreliable, we find that error correcting schemes emerge simply by using noisy units during training; random errors injected during backpropagation result in spreading representations apart. Average and minimum distances increase with misfire probability, as predicted by coding-theoretic considerations. Furthennore, the effect of this noise is to protect the machine against permanent node failure, thereby potentially extending the useful lifetime of the machine.
منابع مشابه
One-point Goppa Codes on Some Genus 3 Curves with Applications in Quantum Error-Correcting Codes
We investigate one-point algebraic geometric codes CL(D, G) associated to maximal curves recently characterized by Tafazolian and Torres given by the affine equation yl = f(x), where f(x) is a separable polynomial of degree r relatively prime to l. We mainly focus on the curve y4 = x3 +x and Picard curves given by the equations y3 = x4-x and y3 = x4 -1. As a result, we obtain exact value of min...
متن کاملFamilies of ternary (t,m, s)-nets related to BCH-codes
A link between the theory of error-correcting codes and (t, m, s)nets leads to the efficient construction of two families of very good ternary nets. These have parameters (4r − 4, 4r, (32r + 1)/2 (for all r ≥ 2) and (2r− 4, 2r, (3r − 1)/2) (for all odd r ≥ 3). The underlying codes are cyclic codes.
متن کاملConstruction of digital nets from BCH-codes
We establish a link between the theory of error-correcting codes and the theory of (t, m, s)-nets. This leads to the fundamental problem of net embeddings of linear codes. Our main result is the construction of four infinite families of digital (t, m, s)-nets based on BCH-codes.
متن کاملLearning efficient error correcting output codes for large hierarchical multi-class problems
We describe a new approach for dealing with hierarchical classification with a large number of classes. We build on Error Correcting Output Codes and propose two algorithms that learn compact, binary, low dimensional class codes from a similarity information between classes. This allows building classification algorithms that performs similarly or better than the standard and performing one-vs-...
متن کاملAn approach to fault detection and correction in design of systems using of Turbo codes
We present an approach to design of fault tolerant computing systems. In this paper, a technique is employed that enable the combination of several codes, in order to obtain flexibility in the design of error correcting codes. Code combining techniques are very effective, which one of these codes are turbo codes. The Algorithm-based fault tolerance techniques that to detect errors rely on the c...
متن کامل